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APOLLO EXPERIENCE REPORT 

COMMAND AND SERVICE MODULE 

SEQUENTIAL EVENTS CONTROL SUBSYSTEM 

By Gary W. Johnson 
Lyndon B. Johnson Space Center 

SUMMARY 


The Apollo command and service module sequential events control subsystem 
provides the automatic timing and control of the critical functions required for the 
launch escape system and service propulsion system aborts, spacecraft stage separa- 
tions, and the Earth recovery system operations. All pyrotechnics (explosive devices) 
on the spacecraft are initiated by this subsystem. 

Major recommendations on the sequencer design include emphasizing the installa- 
tion of flight components in early development hardware, considering the ease of com- 
ponent installation, and implementing the capability for checkout of all redundant 
elements. Because of the heavy emphasis on safety in design and the use of specially 
screened components, no flight failures have occurred in the sequential events control 
subsystem on manned or unmanned Apollo spacecraft. 


INTRODUCTION 


The Apollo command and service module (CSM) sequential events control subsys- 
tem (SECS) is an integrated subsystem composed of the following sequence controllers. 

1. Master event sequence controller (MESC), two 

2. Earth- landing sequence controller (ELSC), two 

3. Lunar docking events controller (LDEC), two 

4. Lunar module (LM) separation sequence controller (LSSC), two 

5. Service module (SM) jettison controller (SMJC), two 

6. Reaction control system controller (RCSC), one 

7. Pyrotechnic continuity verification box (PCVB), one 



The controller operating relationship, the sources of electrical power, and the location 
of the controllers in the CSM crew compartment are given in this report. The SM jet- 
tison controllers are located on the forward bulkhead of the SM, the LM separation 
sequence controllers are located in the spacecraft- LM adapter (SLA), and the RCSC is 
located in the command module (CM) aft compartment. The SECS is used to automati- 
cally sequence time- critical portions of manually initiated normal mission functions. 

A typical logic and pyrotechnic circuit used in the SECS is shown in this report. 

As an aid to the reader, where necessary the original units of measure have been 
converted to the equivalent value in the Systeme International d' Unite's (SI). The SI units 
are written first, and the original units are written parenthetically thereafter. 

CONTROLLER RELATIONSHIP, POWER SOURCES, AND LOCATION 


The interrelationship of the CSM SECS individual controllers and applicable 
sources of power are shown in figure 1. The location of the controllers in the CM crew 
compartment is indicated in figure 2; the SM jettison controllers on the forward bulk- 
head of the SM in figure 3; the LM separation sequence controllers in the SLA in fig- 
ure 4; and the RCSC in the CM aft compartment in figure 5. A typical SECS logic and 
pyrotechnic circuit is shown in figure 6. 



Figure 1.- Relationship of controllers. 
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location. 


2 









Figure 3.- Service module jettison 
controller location. 
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Figure 5. - Reaction control system 
controller location. 



Figure 4. - Lunar module separation 
sequence controller location. 


DESIGN PHILOSOPHY 


The following is an outline of the 
design philosophy applied to the SECS. 

1. The subsystem shall contain no 
single-point failure that will cause either 
premature operation or loss of a function. 

2. The subsystem shall consist of a 
dual- redundant, electrically isolated, sep- 
arately powered, and separately initiated 
system for all functions. 


3. No electrical crossovers shall exist between the two redundant systems except 
through electromechanical contacts (relays). 


4. Failures in one part of the SECS shall not propagate to the redundant part. 

5. The logic control must obtain operating power from a source separate from 
the pyrotechnic power source. 


6. All pyrotechnic devices shall remain electrically shorted until firing. 
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Figure 6. - Typical sequential events control subsystem logic and pyrotechnic circuit. 


7. The subsystem shall have provisions for prelaunch ground support equipment 
(GSE) hardline monitoring. 

8. Logic and pyrotechnic electrical grounding points shall be isolated electri- 
cally and physically. 

9. All power- source negative lines shall be returned to vehicle ground point 
individually. The system shall have no electrical grounds to the vehicle except at the 
vehicle ground point. 

10. Shield circuits shall be grounded inside the controller and at the pyrotechnic 
housing. There must be 360° shielding between the shield and the pyrotechnic case. 

11. The sequence controllers shall be bonded electrically to the spacecraft 
frame. 
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12. All pyrotechnic circuit wiring shall consist of twisted, shielded pairs of 
conductors. 

13. All pyrotechnic connectors shall be keyed so that nonpyrotechnic connectors 
cannot be connected to pyrotechnic devices and so that adjacent pyrotechnic connectors 
cannot be interchanged. 

14. Circuit protection, fault isolation, and current limiting shall be provided in 
all pyrotechnic circuits. 

15. Test points shall be provided for checking all redundant elements and 
circuits. 

16. No flight connectors are to be broken (demated) to perform systems checkout. 

17. Ground support equipment connectors will be provided for checking pyro- 
technic bridgewire resistance after final pyrotechnic installation. 

18. Redundant firing circuits shall be routed through separate connectors; 
redundant functions shall be located in separate controllers; logic circuits shall be 
routed through connectors separate from the pyrotechnic circuits. 

19. All logic timing circuits must fail long, not short (must fail to infinity). 

20. Manual backup switching must be provided on all crew- safety functions. 

21. The two series relays shall be mounted in such a manner that the vibration- 
sensitive axis (actuation plane of contacts) of one relay is orthogonal (90°) to the sensi- 
tive axis of the other relay. 

In a few cases, these criteria were not applied across the board because of space- 
craft volume and schedule constraints. For example, in the case of the PCVB and 
Block II RCSC, the redundant systems were located in the same box. 


DEVELOPMENT HISTORY 


Initially, the sequencing system for the Apollo research and development (R&D) 
flights was to be developed at the NASA Lyndon B. Johnson Space Center (JSC) (formerly 
the Manned Spacecraft Center (MSC)). However, in March 1962, the responsibility for 
the R&D sequencing system was assigned to the Apollo CSM contractor. The initial 
MSC design was a dual- redundant relay design, using flight- qualified components simi- 
lar to those used for Project Mercury. However, the contractor elected to design a 
solid-state logic sequencer with motor- switch power output circuits. 

In January 1964, the MSC Engineering and Development Directorate was given 
Apollo subsystem design and management responsibility and, in March 1964, the solid- 
state sequencer design was reviewed by personnel from the MSC because numerous 
failures occurred during preflight testing of boilerplate 13 (BP- 13). The review 
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revealed that the solid-state design was unnecessarily complex and that numerous 
single failures could cause out- of- sequence functions to occur. It was pointed out that 
changing to a relay design could eliminate the single-point failures, result in the use of 
fewer components, and provide reliable function. For the BP- 13 flight, a relay was 
added to the existing solid-state design to protect the circuit from inadvertent opera- 
tion. Concern was expressed that the operational mission sequencer, which was of a 
solid-state design similar to but more complex than that used in the BP- 13 vehicle, 
would not provide sufficient reliability for the Apollo spacecraft. In April 1964, a sub- 
contractor made a trade-off study of the solid-state design and of a relay design; the 
conclusion was that the relay design would be less complex, more flexible, and more 
reliable. In May 1964, the subcontractor was requested to design, fabricate, and test 
an electromechanical (relay) type MESC. In June 1964, the redesigned mission 
sequencers (MESC and SMJC) were reviewed by the MSC and were judged to be satis- 
factory. However, the ELSC design was not reviewed, and the contractor was requested 
to impose upon the ELSC design all the criteria that were applicable to the MESC design 
(i.e. , no single-point failures). In late July and early August 1964, the CSM contractor 
requested that the subcontractor eliminate the motor switches from the operational 
sequencers. This request was made because of problems that occurred during testing 
(excessive transfer time, low- and high- temperature sensitivity, susceptibility to tran- 
sients, and high contact resistance). On August 20, 1964, the entire sequencing- system 
design was reviewed. At that time, it was decided to move the ELSC from the forward 
compartment and relocate the controllers in the crew compartment, because the 
sequencers were not accessible with the forward heat shield installed. A decision was 
made that test points would be made available to verify final hookup of pyrotechnic 
initiators in the Earth- landing system (ELS). 

It was also decided that an investigation of the single-point failure mode of CM 
switches would be conducted, because failure of a switch could cause premature system 
operation. The investigation disclosed that, according to the preliminary schematic, 
single relays were being used to fire pyrotechnic initiators, which resulted in a single- 
point- failure design. 

The contractor reported that redesigning the ELSC to eliminate the single- point 
failures and incorporating test points (GSE access connector) for final connection veri- 
fication of the initiators would result in large costs and would affect the schedule. 

An interim arm/disarm circuit was added to the existing design so the R&D mis- 
sions could proceed on schedule. This circuit was added to keep the pyrotechnic sub- 
system disarmed during the mission until time to fire pyrotechnic devices, thus 
minimizing the timespan during which potential single-point failures could occur. This 
design concept was incorporated into the R&D mission sequencer that was used at the 
White Sands Missile Range on the boilerplate flights and on spacecraft 9 (SC-009). 

A new junction box was designed that would provide a GSE access connector for 
checkout of the ELS pyrotechnics. The new junction box was designated the PCVB, and 
it operated in series with the ELSC. Use of the new PCVB became effective on SC- 009 
and subsequent vehicles. 

In October 1964, failure problems were experienced during the testing on the R&D 
mission sequencer during vibration exposure. The failures were caused by broken 
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harness wires at relays, a broken internal coil lead to a latching relay, and contamina- 
tion on time-delay relay contacts. The broken coil lead and broken wires at the external 
relay terminations were caused by excessive stress resulting from amplification of 
vibration of the sequencer chassis. The contact contamination was an organic film that 
resulted from solder- flux resin being washed onto the contacts by a cleaning solvent. 
Encapsulation of the relays into the chassis assembly solved the problem of area vibra- 
tion amplification. A burn- in procedure to cycle the relays (break down the contact film 
by the sliding action of the contacts) was used to screen the relays for contact contami- 
nation. Because of these problems, the relays to be used in the operational Apollo 
sequencer were incorporated into the R&D mission sequencer to gain test and flight 
experience early in the program. 

During vibration testing (April 1965) of the mission sequencer that contained the 
operational relays, contact resonance occurred and caused unacceptable contact chatter. 
The manufacturer of the relay resolved this problem by a design change that involved the 
installation of a vibration damper in the relay. The discovery and correction of the 
relay faults occurred before the problems could affect operational hardware. The effort 
required to resolve the problems caused by relay design emphasizes the need to qualify 
hardware as soon as possible during the R&D phase of a program. Design problems, 
associated with the GSE pyrotechnic simulators that caused failure of flight sequencers 
because of overload conditions, were discovered during checkout of the boilerplate vehi- 
cles. The design problem that caused the overload condition was the use of normally 
closed relay contacts (closed when the relay coil is unpowered) to apply the simulated 
pyrotechnic load. Loss of facility power to the GSE prevented the current- sensing cir- 
cuit from interrupting the load (applied by the deenergized relays). The GSE was 
redesigned to remove the load (to be fail-safe) when electrical power was removed. 

All Block I operational Apollo sequence controllers were qualification tested dur- 
ing 1965. Two models of each type of controller were subjected to qualification testing. 
One unit was exposed to design proof testing, thus verifying that design parameters had 
been met. The other unit was mission simulation tested (one operational cycle and one 
subsequent mission cycle at nominal mission conditions) and then subjected to off- limit 
(extreme stress) testing at levels not exceeding 2. 5 times the design proof level. 

Six major problems were encountered during qualification testing. 

1. Long timeout of the time delays in the first cycle (after long-term storage or 
exposure to high temperature) was caused by impurities in the timing capacitor. The 
problem was resolved by using a different type of capacitor and screened parts. 

2. Pyrotechnic- firing relay contacts became welded closed when interrupting cur- 
rent (because the normally closed contact was connected to ground). The problem was 
resolved by using a relay in which the normally closed contact was ungrounded to inter- 
rupt the current. 

3. Silver particles inside a stud- mounted diode caused a short to the mounting 
plate. To screen out the silver particles causing the electrical shorts, all stud- mounted 
diodes were X-rayed. 
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4. Misalined relay contacts caused the relay to fail. New weld process controls 
and inspection were incorporated to prevent relay contacts from being misalined during 
welding. 

5. Hangup of a baroswitch plunger prevented the baroswitch from operating. 
Tighter dimensional limits were established for the baroswitch plunger and outer case 
clearance to prevent this hangup. 

6. Moisture under a conformal coating resulted in a decrease in the insulation 
resistance. The conformal- coating/humidity problem was resolved by using a silicone 
rubber coating on the components and wiring in the sequence controller. 

In January 1965, a redesign of the Block II sequencing system was proposed to 
consolidate all the sequencing functions into two controllers per system (one in the CM 
and one in the SM) instead of the five controllers per system used in Block I. The new 
design would incorporate the added Block II mission functions (docking) and would elim- 
inate the single-point failures and checkout limitations in the ELSC. The new design 
would involve the same components (relays, time delays, etc. ) and would save weight 
and space. For future hardware (part of Block II), a study was proposed to investigate 
the feasibility of solid-state component mechanization of the sequencing system. In 
June 1965, the proposal was turned down on the basis that the experience gained on the 
Block I hardware would provide greater assurance of reliability than that to be gained 
through simplification and improvements of the system if the system were redesigned. 

A decision was made to eliminate the single-point failures in the Block I sequencing 
system and to design the Block II functions into new assemblies, so that the Block I 
equipment would be changed as little as possible in the transition from Block I to 
Block II spacecraft. In July 1965, it was decided to eliminate the single-point failures 
in the ELSC by adding series relays to the PCVB. The additional Block II functions 
were to be performed by two new controllers, the LDEC and the LSSC. 

In May 1965, a review of the electrical schematics, during the SC- 009 design 
engineering inspection, revealed that a single-point failure of a connector (P/J7) on the 
MESC would give an inadvertent automatic emergency detection system (EDS) abort. 

A jumper was provided to eliminate the connector single-point failure. 

In December 1965, a manual switch failed to operate during the thermal- vacuum 
qualification test. Because of this failure, a change was approved in January 1966 to 
add redundant panel switches on manually initiated crew- safety functions on all manned 
Apollo spacecraft. 

On May 19, 1965, the need for an automatic abort system was demonstrated 
dramatically during the BP- 22 flight at the White Sands Missile Range. The launch 
vehicle disintegrated because of excessive roll when a control surface failed hard- over 
during launch (figs. 7 and 8). When the abort- initiate wires broke in the launch vehicle 
and deenergized the CM abort relays, the abort was initiated automatically. This prob- 
lem highlighted the usefulness of a hot-wire abort system (an opening in wires powered 
from the spacecraft initiates the abort) to detect launch vehicle structural failure and 
safely abort the flight and recover the spacecraft. 
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Figure 7. - Launch vehicle breakup Figure 3. - Launch escape motor pulls 

initiates abort. command module to safety. 


In January 1966, during plugs- in testing on SC-009 at the NASA John F. Kennedy 
Space Center (KSC), a failure occurred on the ELSC main- para chute- deploy relay. The 
GSE pyrotechnic simulators were overloading the relay contacts during checkout. This 
overload caused the relay contacts to weld closed and allowed the main-parachute simu- 
lators to fire when the ELSC pyrotechnic bus was armed. Both the GSE and the 
sequencer system circuits were changed to eliminate this problem. The GSE was made 
compatible with the actual spacecraft circuit- loading requirements, and the sequencer 
circuits were modified to reduce the loading on the ELSC relay contacts. Testing was 
performed to verify that the new design could meet or exceed the mission requirements. 

On February 26, 1966, SC- 009 (Apollo/Saturn 201) was launched. This was the 
first flight for the Block I SECS (consisting of the MESC, SMJC, ELSC, RCSC, and the 
PCVB). Only the ELSC and PCVB had been flight tested previously (ELSC and PCVB 
on SC-002 and ELSC also on BP- 22). Because the vehicle was unmanned, the non- 
standard Block I equipment flown included a postlanding sequence controller, an impact 
switch, and a fuel- dump box. 

During the entry phase of the mission, the circuit breaker that powered the 
sequencer system B relays (arming the logic and pyrotechnic bus) opened. This occur- 
rence caused a loss of power to the system B sequencers and resulted in the loss of 
system redundancy. The remaining system A performed the required Earth- landing 
and postlanding functions satisfactorily, except for the main- para chute disconnect and 
CM reaction control system (RCS) system B propellant dump. The latter two functions 
required both systems to be operational. Postflight failure analysis revealed that a 
wire (associated with a never- implemented SM thermal control system) was powered by 
the system B circuit breaker and was not deadfaced at CM-SM separation. During 
entry heating, the wire shorted and caused the circuit breaker to open, which caused 
loss of function. The wire had not been disconnected or removed when the thermal con- 
trol system was eliminated, and the latest vehicle drawings did not show the wire. 
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This problem underscored the need to disable unused, powered circuitry and to have 
vehicle drawings reflect the actual vehicle configuration (both functional and nonfunc- 
tional). From January 1966 to July 1966, the sequencer hardware was qualification 
tested in support of the manned Block I and Block II spacecraft requirements. The 
hardware unique to the Block II mission requirements was qualification tested from 
August 1966 to December 1966. 

On August 25, 1966, mission AS- 202 (SC-011) was launched, and no problems 
were experienced on the sequencer system during the mission. The sequencer hard- 
ware was the same as that used on SC- 009, except that the PCVB contained the relays 
to eliminate the ELSC single-point failure. During the first (August 1966) and second 
(October 1966) manned thermal-vacuum tests of SC- 008, the system B MESC did not 

2 

provide the CM-SM separation functions under a 34 474- N/m (5 psi) environment, but 

2 

it did function at 101 353 N/m (14. 7 psi). The failure was repeated on the MESC in a 
vacuum chamber at the vendor facility. The circuit performed in the manner of a baro- 
switch; that is, it opened at altitudes greater than 7010 meters (23 000 feet) and closed 
at altitudes less than 5791 meters (19 000 feet). Subsequent failure analysis revealed 
that the problem was an inadequate solder joint at a diode terminal. The expansion of 
the flexible potting in the diode module (cordwood construction), caused by entrapped 
air bubbles, separated the diode from the solder terminal. This open circuit prevented 
the CM-SM separation relays from energizing. The fault was corrected by the use of a 
hard and relatively inflexible encapsulating material and better quality control on solder 
connections. This failure emphasized the need for screening flight hardware by testing 
electrical hardware functionally under spacecraft- environmental conditions. 

In January 1967, during testing of RCS engines on SC-012, the system B SMJC 
did not energize the plus- roll SM RCS jets (5. 5- second time delay timed out in 0 sec- 
ond). Failure analysis revealed a shorted output transistor in the time delay. Anal- 
ysis of the transistor indicated failure was caused by high-voltage punch- through 
(negative- voltage transient exceeded transistor rating). Tests indicated 500- to 
600- volt negative spikes from the RCS engine solenoids (inductive kickback) during 
engine firings. A diode was added to the SMJC 5. 5- second time delay to prevent tran- 
sients from damaging the output transistor. Arc- suppression diodes could not be added 
to the engine solenoids because doing so would increase the minimum pulse time when 
the engine was fired. 

After the SC- 01 2 fire in January 1967, flammability tests were conducted on an 
2 

MESC in a 110 316-N/m (16 psi) oxygen environment. Ignition of the room- 
temperature- vulcanizing (RTV) encapsulant on a wire, harness inside the box resulted 
in self- extinguishment of the fire. 

The Block II RCSC underwent qualification testing from January 1967 to 
March 1967. Numerous problems were experienced with the relays (contact chatter) 
and with the time delays (long timeout) during vibration testing. The failure analysis 
revealed that the components received excessive vibration caused by amplification in 
the chassis. In addition, the testing determined that the relay in the assembly was not 
designed for the test- vibration levels used and that the time- delay capacitor seals 
became deformed under vibration and caused electrolyte leakage (long timeout). 
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The RCSC was designed by the electrical group, even though it was considered a 
sequencer (fired pyrotechnics). All the other sequencers in the system were designed 
by the sequencer group. As a result, the early R&D flight experience gained on the 
sequencers was not applied to the RCSC. The vibration- amplification problem experi- 
enced in the R&D program and the ELSC vendor design experience resulted in all relays 
being encapsulated in the units designed by the sequencer group. These sequencers also 
used flight-proven high- reliability relays and time delays. Because of the failure of a 
relay used in Block I to switch three-phase alternating- current power, the electrical 
group used a new relay design in the Block n hardware. This new relay was used in the 
Block II RCSC in place of the flight-proven relays used by the sequencer group. 

The corrective action taken to resolve RCSC relay problems included the encap- 
sulation of the relays and time delays in the circuit, the replacement of the electrical- 
group- designed relay with a sequencer- group- designed relay, and the vibration screening 
of the capacitors used in the time delays. This problem illustrates the need to have a 
common design philosophy and to apply the experience gained from the development 
testing throughout the subsystem. 

In September 1967, during integrated testing of SC-020 at the contractor site, a 
system A SM minus- roll engine circuit shorted when the engine was turned off. The 
problem was traced to a stud- mounted diode in the RCSC; the 0. 005- centimeter 
(0. 002 inch) thick mica washer had been damaged during installation. Later, during 
repeated application of high-voltage transients caused by the inadvertent removal of the 
ground from the arc- suppression network, a breakdown occurred at the mechanically 
weakened point. The corrective action taken on Block II was to install a thicker mica 
washer (0.013 centimeter (0.005 inch)) to give added mechanical and dielectric strength 
and to modify the diode- installation procedures. 

The problem could be partially attributed to the original installation design, in 
which a very thin and fragile mica washer was used to obtain better heat transfer than 
was necessary. No heat- sinking problem existed in the minus- roll engine circuit; 
therefore, it would have been wiser to use a thicker, less fragile washer that could 
withstand some handling without damage. This fact exemplified the need for a designer 
to consider not only what provides the best performance, but also the ease of assembly, 
the durability, and the maintainability of the system. 

On November 9, 1967, the Apollo 4 mission (SC-017) was flown. No problems 
were experienced in the sequential system. The in-flight telemetry did not indicate all 
failures in redundancy; therefore, to ensure that no failures had occurred, postflight 
tests were run on all sequence controllers. These tests verified the absence of failures. 

In January 1968 at the KSC, the 2. 0- second time delay in the system A SMJC on 
SC- 020 failed at initial turnon (no time delay) during integrated testing. The failure 
analysis revealed that the output transistor was shorted by a high-voltage negative tran- 
sient. Testing revealed that the transient was generated when the circuit breaker was 
opened on a GSE box that powered the SMJC. Opening this circuit breaker deenergized 
the SM RCS engine simulator, which generated a much greater kickback voltage tran- 
sient than the engine solenoids. This transient, coupled with the opening of the circuit 
breaker which removed the bus arc-suppression network, resulted in a voltage tran- 
sient that exceeded the 18 5- volt capability of the SMJC circuit transistor. The 
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corrective actions taken were adding arc- suppression diodes to the GSE circuit- breaker 
box and converting the RCS engine simulator (A14-275) to a purely resistive load. 

During this period, two design changes were made to the sequential system on 
CSM-101 and subsequent vehicles. A crew- safety- critical single-point failure existed 
in the launch-escape-tower-jettison circuit. If a short to ground occurred near the 
tower leg connector in the negative return of the abort- mode relay, the shorted system 
would not switch from the launch escape system to the service propulsion system (SPS) 
abort mode. If an SPS abort were initiated in this abnormal condition, premature 
CM-SM separation would occur. The corrective action was to replace the two- position 
panel switch with a three-position switch to provide a redundant method of switching the 
relay to the SPS abort mode. The second change was providing additional GSE connec- 
tors for checkout and separating the pyrotechnic and logic wiring leading to the PCVB 
and LDEC. This correction made it possible to reverify the logic circuits when the 
spacecraft connector was disconnected to check pyrotechnic bridgewire resistance. 

In February 1968, the Configuration Control Board approved the addition of circuit 
breakers on CSM-101 and subsequent vehicles to isolate the pyrotechnic bus from the 
main- parachute- disconnect pyrotechnic circuit. Isolation prevents premature discon- 
nection of the main parachutes caused by a crew- safety- critical single-point failure of 
the panel switch. On April 4, 1968, the Apollo 6 mission (SC- 020) was flown. The 
sequential systems performed all the required mission functions satisfactorily. 

In May 1968, at the CSM-101 phase III Customer Acceptance Readiness Review, 
the MSC indicated that the A14-275 RCS engine simulator at the KSC had not been modi- 
fied to prevent SMJC damage. Because the simulator inductance function was needed, 
the board decided to install switches in the A14-275 to allow the KSC to enable or disable 
the inductive load portion of the simulator. All the simulators at the contractor facility 
had the inductance disabled. 

However, in June 1968, the SM jettison controllers on CSM-101 at the KSC had to 
be replaced because the arc- suppression network was installed on the wrong side of the 
circuit in the GSE circuit breaker box. The GSE box was reworked to the proper 
configuration. 

In July 1968, during preparation for the CSM-101 unmanned chamber test at the 
KSC, a 5- ampere fuse in the system A MESC was found to have been blown. Trouble- 
shooting revealed that a shorted wire in the translation hand controller caused the blown 
fuse. The fuse was replaced. During the MESC retest, wires that had been demated 
and then remated were found broken at the pyrotechnic connectors. This experience 
points out one of the hazards that must be considered when hardware is removed from 
the spacecraft and emphasizes the need to reverify the integrity of every wire associated 
with connectors that have been disconnected. 

Spacecraft alarms occurred when the CM RCS heater switch was turned off during 
the manned chamber test on CSM-101 at the KSC and at the integrated tests at the con- 
tractor facility (CSM-104). Investigations revealed that the alarms were caused by 
deenergizing the CM RCS engine solenoids. According to test procedures being used at 
that time, when the CM RCS heater switch is turned on and off, the CM arc- suppression 
network is disabled. The problem was resolved by a procedural change that would 
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prevent the problem during ground checkout. This change was transferring the engine 
logic (including the arc- suppression network) to the CM RCS before turning off the 
heater switch. Later, an arc- suppression network was installed on the CM RCS cir- 
cuitry for CSM-108 and subsequent vehicles. 

Block II sequential system hardware was initially flown on the first manned Apollo 
flight (Apollo 7) on October 11, 1968. No problems were experienced with the sequen- 
tial systems. The Apollo 7 crew reported that the automatic sequencing of the para- 
chutes occurred at the proper altitude; no manual backup switches were used. The 
commander reported a delay in disconnecting the main parachutes after splashdown 
because he had closed the switch guard on the ELSC logic switch before impact, being 
concerned that his knee might hit the guard at that time. The system is so designed 
that closing the switch guard will put the switch in the open position, which removes 
power from the ELSC and recycles the 14- second time delay. The ELSC logic switch 
was turned on after splashdown, but it did not activate until after the 14- second time 
delay to time out. The delay in disconnecting the parachutes could have been the reason 
the spacecraft turned over (stable II position) after landing. 

On December 21, 1968, the Apollo 8 mission (CSM-103) was flown without any 
anomalies noted in the performance of the SECS. However, a change in the crew launch 
procedures for the sequencer system had been made for this mission. During launch, 
the commander opened the system A and B SECS ARM circuit breakers after tower jet- 
tison to disarm the CM-SM separation toggle switches. The reason for the procedural 
change was to prevent a premature CM-SM separation in the event of the failure of a 
crew- safety toggle switch. Failures that could possibly cause premature operation had 
occurred in similar toggle switches before the flight. A contact button could be loose 
in the switch, bridging the gap in the open contacts and energizing the function prema- 
turely. The CM-SM separation switches are classified as crew- safety- critical after 
tower jettison; therefore, that was the time chosen to disarm the switches. The fail- 
ures that had occurred were too close to the launch date for the switches to be replaced, 
so the procedural change was made. On subsequent spacecraft, the crew- safety 
switches were screened by means of X-rays for evidence of loose contact buttons. 

Additionally, in late 1968, an SM overcurrent motor switch in the electrical power 
system failed during the altitude chamber test on CSM-104 at the KSC. Investigations 
revealed that the failure was caused by transients generated by the A14-275 RCS engine 
simulators. In January 1969, the CSM-104 Flight Readiness Review board directed the 
contractor to modify the simulators to prevent the transient problem. This time, all 
simulators were modified to present a resistive load. This problem is an example of 
how neglecting to correct a problem in test hardware quickly, properly, and at all 
phases of spacecraft testing resulted in flight hardware being damaged. Not all units 
had been modified as long as a year after the first direction was given to modify the 
A14-275. 

During the checkout of LM-4 and CSM-106 LSSC circuitry in February 1969, an 
LM-SLA tension- tie circuit was crosstied with the guillotine circuit in the LM. This 
crosstie resulted in a much longer wire run for the pyrotechnic- guillotine circuit. The 
problem would not have been discovered if an old test battery that had a lower-than- 
nominal output voltage had not been used. The reduced output voltage, coupled with the 
higher resistance path (a longer wire), resulted in the pyrotechnic simulator box not 
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receiving sufficient current to present a GO indication. Investigations revealed that the 
problem was caused by miswiring. The miswiring was caused by lack of a clear defini- 
tion of the interface in the interface control document (ICD). The ICD could be inter- 
preted to indicate that both the LM and the CSM were wired correctly. Corrective 
action was taken to interchange wires at the system A and B LSSC connectors. This 
problem indicated the need to test spacecraft circuits, not only at nominal voltages but 
also at design-limit voltages, to verify that the vehicle (as built) will meet the design 
requirements. Also, in an electrical connector interface ICD, every pin should be 
identified clearly on each side of the interface. 

On March 3, 1969, the Apollo 9 mission (CSM- 104) was flown; this mission was 
the first on which the lunar- mission functions were performed by the LDEC and the 
LSSC. All the sequential system functions were performed satisfactorily during the 
mission. 

On May 18, 1969, the Apollo 10 mission (CSM- 106) was flown and resulted in the 
successful operation of all sequential functions. Some procedural changes were made 
on this flight because of concern over having the sequencer system armed at times when 
it was not to be used. During launch, the ELS switch was in the automatic mode for the 
first 30 seconds of the mission to have the automatic ELS capability for near-pad abort. 
Then, this switch was placed in the manual position. For entry after CM-SM separa- 
tion, the pyrotechnic bus was to be disarmed, then rearmed at an altitude of 
15 240 meters (50 000 feet). Then, the ELS switch would be in the automatic position 
at an altitude of 9144 meters (30 000 feet). 

In May 1969, when a review of the sequential system -was being conducted by the 
MSC to locate terminal boards of a configuration on which failures had occurred (ref. 1), 
it was discovered that two EDS abort signals were passed through the same spacecraft 
electrical connector and that two booster- engine- cutoff commands went through another 
single connector on the MESC. These connectors would be single-point failures that 
could cause an abort of the mission. Clearly, this situation was a violation of the EDS 
design subpanel ground rule for routing redundant EDS functions through separate con- 
nectors. This ground rule was established in November 1966 for the Block II CSM EDS. 
The corrective action was to safety- wire the connectors together on CSM- 107 and 
CSM- 108 to prevent inadvertent disconnection. On CSM- 109 and subsequent vehicles, 
these critical functions were routed through separate connectors. 

In July 1969, during acceptance testing of an MESC, the apex- cover- jettison func- 
tion failed to occur. Investigations revealed that one pin of a fuse- module connector 
was not seated properly. Analysis of the fuse-module connector-assembly tolerances 
indicated that a marginal pin/socket engagement condition could exist (fig. 9). Cor- 
rective action was taken to remove two flat washers and to insert a thinner O-ring 
(0. 124- centimeter (0.049 inch) compared with 0. 178- centimeter (0.070 inch) or 
0. 157- centimeter (0. 062 inch) thick (fig. 10)) to provide additional engagement distance. 
This change was effective on CSM- 108 and subsequent vehicles. 

On July 16, 1969, the first lunar- landing mission, Apollo 11 (CSM-107), was 
launched; no problems related to the SECS occurred. The only procedural change made 
was that the ELS logic switch was set on manual after tower jettison rather than at 
T + 30 seconds as had been done on the Apollo 10 launch. This change was a crew- 
preference revision to the launch procedures. 
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Figure 9.- Master event sequence con- Figure 10.- Master event sequence con- 
troller fuse module, original design. troller fuse module, after modification. 


In August 1969, a change was made to power the crew- safety CM-SM separation 
switches from the ELS circuit breakers (instead of from the SECS ARM circuit break- 
ers) on CSM-109 and subsequent vehicles. This change would allow the switches to be 
disarmed, except during the launch and entry phases of the mission. 


At that time (August 1969), a post- 
flight SM trajectory analysis revealed that 
the SM maneuvers after CM-SM separation 
were not satisfactory for preventing pos- 
sible CM-SM recontact during entry. In 
November 1969, a directive was issued to 
modify the SMJC (fig. 11). This modifi- 
cation consisted of replacing the 
5. 5- second plus- roll engine time delays 
with 2. 0- second time delays and adding 
25- second time delays to turn off the 
minus-X RCS engines. 

During the Apollo 12 launch 
(CSM-108) on November 14, 1969, the 
spacecraft was struck by lightning, and 
the spacecraft electronic systems were 
affected. Primarily, the overload- 
protection electronics equipment that 



Figure 11.- Service module jettison 
controller changes. 
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involved a silicon- controlled rectifier was affected. The electromagnetic interference 
protection designed into the SECS proved to be effective because no inadvertent firing of 
pyrotechnics occurred. 

In December 1969, a decision was made to eliminate the crew- safety- critical 
single-point failures on the SECS backup pushbutton switches by wiring contacts in 
series for each system on the Skylab CSM. In January 1970, the same change was 
implemented for the J-mission vehicles (CSM-112 to CSM-115A). 

During the electrical mating EDS test on Apollo 13 at the KSC, the spacecraft/ 
launch vehicle/GSE loop went into an oscillatory mode and cycled the automatic- abort 
logic relays of the vehicle approximately 1200 times before the test operator removed 
test power. Investigations revealed that a change to the previous EDS test was made 
for this vehicle and that the tower- jettison switches were placed in the off position 
rather than in the automatic position used in the previous EDS test. This switch posi- 
tioning resulted in the launch vehicle test- abort command being routed to the SPS abort 
logic, which energizes the SLA separation pyrotechnic relay, removing the spacecraft- 
pyrotechnic- relays- indicate- safe signal to the launch vehicle GSE. Accordingly, the 
launch vehicle GSE removed EDS test power (which powered the simulated abort stim- 
ulus). However, the EDS test power is not latched off; thus, the loop oscillates at a 
frequency determined by the characteristics of the relays and time delays of the space- 
craft, launch vehicle, and GSE (approximately 70 msec/cycle). Corrective action was 
taken to eliminate the procedural change made on the tower jettison switch position and 
to modify the launch vehicle GSE to latch the test power off when the safe indications 
had been interrupted. The number of cycles on the MESC relays was far below the 
rated life; therefore, no hardware changes were made. This incident showed that, 
when a test procedure has been used successfully on previous vehicles, any changes 
must be reviewed very carefully by everyone concerned with the affected system. 

On April 11, 1970, the Apollo 13 mission (CSM- 109) was launched. The planned 
lunar landing had to be aborted because of an abrupt loss of SM cryogenic oxygen. 
Because this loss necessitated retention of the LM as long as possible, the sequence of 
events before entry had to be changed. Instead of the LM being separated from the 
CSM, the CM and LM were separated from the SM, then the CM was separated from the 
LM before entry. The remaining functions were performed in the normal sequence. 

The missions flown with the scientific instrument module (SIM) bay located in the 
SM (J-missions) required firing pyrotechnics to jettison the SIM- bay door, to launch 
the subsatellite (from CSM-112 and CSM- 113), and to jettison the high-frequency 
antenna (from CSM-114). The relays for performing these functions were incorporated 
into the multiple- operations module (MOM) box, which is located in the upper SIM bay 
(fig. 12). The MOM box was primarily a power- distribution box for the SIM- bay exper- 
iments but, because pyrotechnic firing relays also were involved, the MSC sequencer 
subsystem manager followed the development of the MOM box. The MOM box under- 
went qualification testing from September to December 1970. 

During vibration testing, one of the redundant relays that switches alternating- 
current power to the experiments failed to operate. Investigations revealed that the 
armature- return spring had broken. Microscopic inspection revealed the break to be 
adjacent to a tool mark. The break pattern indicated that the fracture resulted from 


16 



Figure 12.- Multiple- ope rations module 
box location. 


material fatigue. A design- evaluation 
vibration test was conducted on the relays 
to determine if the failure was caused by 
extended vibration exposure or by high 
levels of vibration (or both). Based on this 
testing, it was concluded that the failure 
was caused by an isolated quality defect 
(the tool mark); however, to ensure that 
marginal relays were not installed in the 
MOM box, a relay- component vibration- 
screening test before installation was 
implemented. 

For crew safety, relays were added 
to the LDEC to isolate the SIM- bay pyro- 
technic bus from the spacecraft pyrotechnic 
bus. Then, the SIM- bay bus could be armed 
in flight without arming the spacecraft 
pyrotechnics. 

On January 31, 1971, the Apollo 14 
mission (CSM-110) was flown. The only 
major change in the operation of the sequen- 
tial system was that the crewmen preferred 
to launch with the ELS logic switch in man- 
ual, keeping the ELS baroswitches disarmed. 


On July 26, 1971, the Apollo 15 mis- 
sion (CSM-112) was flown. This mission 
was the first flight with the SIM bay. All the required sequential functions were per- 
formed successfully, including the new pyrotechnic functions for SIM- bay-door jettison 
and subsatellite launch. The main- para chute anomaly that occurred during CM RCS 
propellant burn- off did result in the RCSC CM- RCS dump- inhibit timer being changed 
from 42 to 61 seconds for CSM-113 and CSM-114. The Skylab CSM RCSC already con- 
tained the 61- second timers because of the use of the Saturn IB launch vehicle. 


During the initial sequencer- subsystem checkout at the contractor facility in 
September 1971, some EDS functions were not received from the MESC on the first 
Skylab vehicle (CSM- 116). Investigations revealed that the spacecraft wire changes 
(for eliminating EDS connector single-point failures) that were approved in May 1969 
were not implemented on the Skylab CSM- 116 to CSM- 119. The corrective action was 
to incorporate the approved wire change. This problem was identified originally on the 
SC- 009 Block I vehicle and was to have been corrected on all subsequent vehicles. 
However, each time a major change in spacecraft vehicles occurred, Block I to Block II 
and Block II to Skylab, the authorized change was not implemented. This situation 
emphasizes the need to ensure that approved changes are carried over from one vehicle 
to the next, even though these changes are approved for incorporation on later vehicles. 
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In December 1971, an Apollo sneak- 
circuit bulletin for CSM-113 identified a 
sneak circuit in the ELSC and PCVB that 
could possibly result in premature deploy- 
ment of the parachutes. If the single 
ground connection to the ELSC should fail 
open, then a sneak circuit would exist 
(fig. 13) that could result in deploying the 
drogue and pilot parachutes under the apex 
cover and cause loss of the crew. The cor- 
rective action taken was to add a redundant 
ground wire to the ELSC on CSM-113 and 
subsequent vehicles. 

This crew- safety hazard, which was 
discovered by the sneak- circuit group, 
underscores the need for sneak- circuit 
analyses to be run on all critical spacecraft 
circuitry. Also pointed out is the unseen 
hazard that can exist when wiring modifica- 
tions are made external to an electrical box 
to add additional components. This addi- 
tion nullifies the original design of the box 
and allows circuit paths to exist that the 
original designer had never planned. In 
this case, relays were added to the PCVB. 

Rather than redesign the ELSC with the 
added relays incorporated (which would 
have been more desirable), it was decided 
to tie the wiring into the ELSC external to 
the controller. 

Also in December 1971, circuit breakers were added to the LM docking ring 
final- separation circuit for CSM-113 and subsequent vehicles. The addition of these 
circuit breakers was the result of reliability upgrading of the manual LM final- 
separation switches to crew- safety criticality for the premature-failure mode. The 
added circuit breakers downgraded the criticality. This action shows the importance 
of reliability upgrading to ensure that the correct criticality is placed on the spacecraft 
components early in the program. Any later change could result in expensive design 
changes to the spacecraft. 

On April 16, 1972, the Apollo 16 mission (CSM-113) was launched and the SECS 
performed all the required mission functions successfully. The only operational change 
that was made in the SECS for this mission was to leave the pyrotechnic buses armed 
after CM-SM separation to prevent undetected motor- switch failure that would cause 
the loss of one pyrotechnic system for the ELS. 

On December 7, 1972, the last Apollo lunar mission (Apollo 17; CSM-114) was 
launched and the SECS performed flawlessly throughout the flight. The only changes in 
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Figure 13.- Earth- landing system 
sequence controller sneak circuit. 
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the SECS from the Apollo 16 mission were the deletion of the subsatellite- launch pyro- 
technic function and the addition of the high-frequency-antenna-jettison pyrotechnic 
function. 


RECOMMENDATIONS FOR FUTURE SPACECRAFT PROGRAMS 


As a result of the development and flight experience gained during the Apollo 
Program and experience from related studies, the following recommendations are 
made for future spacecraft programs. 


Component Recommendations 

Early in a program, major emphasis should be placed on obtaining reliable com- 
ponents. One- hundred-percent testing, both environmental and electrical, should be 
accomplished on the components. The required performance capability should allow 
for a margin oyer expected design conditions, which will help avoid a costly redesign or 
change in components if unforeseen changes in the component environment occur later. 
Installing the operational components in the developmental hardware allows for the dis- 
covery and correction of component faults before the problems could have an effect on 
the operational hardware. 


Sequencer-Design Recommendations 

The ease of installation and the accessibility for rework should be considered in 
the sequencer design. Connectors should be keyed to prevent inadvertent interchange, 
and sufficient room should be provided between connectors for easy mating and demating 
after installation of the sequencer in the spacecraft. Fuses should be easily replace- 
able without disconnecting or removing the controller. Internal circuitry should be pro- 
tected from electrical transients and reverse polarity. When logic power is routed 
from one sequencer to other sequencers, fusing and diode isolation should be provided; 
these provisions prevent loss of the sequencer- logic bus because of external shorts and 
prevent sneak circuits caused by loss of ground in external hardware. Timing for time- 
delay functions should be capable of being changed without major modifications to the 
sequencer. When electromechanical components (such as relays) are installed, they 
should be potted into modules to minimize vibration- amplification problems caused by 
the sequencer assembly. In designing the installation of stud- mounted diodes, ease of 
assembly should be considered to prevent damage to the fragile mica washer. 


Test and Checkout Recommendations 

The capability for test and checkout of all redundant elements must be designed 
into the hardware. It is desirable to have the capability to verify (after installation) all 
redundancies in the spacecraft without breaking flight connections. 

In testing, emphasis should be placed on development testing to uncover problems 
early in the program, rather than waiting to discover the problems during qualification 
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testing. End-to-end continuity and isolation tests should be performed on the subsystem 
(with the GSE connected) before power-on testing, ensuring the proper configuration of 
the spacecraft and the GSE and saving time later when power-on troubleshooting of the 
vehicle is accomplished. The subsystem testing should also ensure that the redundant 
systems actually do work independently of each other. The GSE used for checkout must 
have sufficient resolution to verify that the subsystem meets the required criteria. A 
failure in the GSE should not damage the spacecraft hardware. 

CONCLUDING REMARKS AND RECOMMENDATIONS 


No failures have occurred on the sequential events control subsystem during flight 
throughout the Apollo Program, including all research and development flights. One 
reason for this performance is the concern for the crew- safety aspect of the sequential 
system, which resulted in much attention being directed both by NASA and contractor 
management to the system and its problems. This same attention should be applied to 
systems that involve pyrotechnic functions in future spacecraft. 

The major recommendations for providing a reliable sequential events control 
subsystem on future spacecraft include the screening and early testing of components, 
the protection of internal sequencer circuitry from electrical transients, overload and 
isolation protection on circuits routed external to the sequencer, and provisions for 
checkout of all redundant elements. 


Lyndon B. Johnson Space Center 
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Houston, Texas, January 17, 1975 
914-11-00-00-72 


REFERENCE 

1. White, Lyle D. : Apollo Experience Report — Electrical Wiring Subsystem. 
NASA TN D-7885, 1974. 


I 

i 


20 


NASA-Langley, 1975 


S-436 


